An Ordered Logit Model of Air Traffic Controllers’ Conflict Risk Judgment

Though there is a considerable agreement amongst past studies about the great variability in conflict judgments by Air Traffic Controllers (ATCos), certain behaviors observable in control rooms speak in favor of the existence of a common core they would that controllers share regarding conflict risk perception. The study presented in this paper began with the construction (from real recordings) of traffic scenarios showing two converging aircraft in approach. Three variables characterized these traffic scenarios, quantifying respectively horizontal separation, vertical separation and the momentum of the formation of the judgment (prediction time span). The conditions created by factorial manipulation of three variables led to the design of short scenarios (about 1min) involving two aircraft upon which 161 controllers gave their judgments about possible occurrence of a separation loss. A first descriptive analysis of the data, conducted in Averty (2005), confirmed the large variability of the experts’ judgments but also clearly indicated the global consistency of the results. The data thus called for a deeper statistical analysis, the results of which will be presented in the article. In a first step, particular models have been constructed for each value of the prediction time span. The comparison of the model’s parameters allows evaluating the influence of the time span on the conflict perception. It appears for example that the horizontal dimension has more "separating power" that the vertical dimension far from the conflict location, but that its relative importance diminishes (along with uncertainty) as the conflict resolves. Individual models are then nonlinearly aggregated into an "integrated model" by maximum likelihood estimation on the whole dataset. Finally, the relevance of this model to individual models is statistically validated, indicating that very few information has been lost in the aggregation process. Introduction Air Traffic Control (ATC) is composed of several tasks that generally overlap over time. The conjunction of these, especially in heavy traffic, may result in the controller's processing capacities being exceeded. Global agreement exists for aiding air traffic controllers (ATCos) by providing them with automated tools that would perform some of the most demanding subtasks. Among these, the Conflict Detection and Resolution (CD&R) processes are traditionally singled out. This sharing of tasks in real time between operator and system requires a sufficient knowledge of experts’ key processing mechanisms, so as to comply with safety and capacity target levels. In this study, we address the conflict detection process and provide a predictive model of ATCos conflict judgments. Relevant research in the domain of conflict detection can be categorized into two fields that include automatic detection schemes based on engineering intuition (CD&R systems) as reviewed in Kuchar and Yang (2000), as well as controller centered literature that aims at understanding the cognitive processes involved in the detection task and favors the investigation of factors that affects controllers performance (Leroux, 1999) (Nunes and Mogford, 2003). The CREED project (Conflict Risk Evaluation based on Expert Detection) undertaken at the DSNA stems from these two fields as its objective is to provide an expert model of Conflict Detection, that is, a model that will have as output the expected conflict risk perceived by controller rather than a conflict/non conflict diagnosis based on objective threshold metrics as in usual CD&R systems. To this end, a large experiment was conducted in which experienced controllers, confronted with pairs of converging aircraft in an approach environment, were asked to provide judgments about possible occurrence of a separation loss. In this article we analyze the collected data and present a statistical model that predicts the controllers' judgments for the conflict geometries considered in the experiment. Our contribution is twofold. First, we show how information gathered from the cognitive literature on conflict detection can be exploited to describe the conflict geometry in a way that remains close to the mental representation of ATCos. Then, we show that this description can be used further to build a statistical model that performs well in predicting ATCo conflict risk judgments for the class of conflicts considered in the CREED experiment. The paper is organized as follows. The first sections present a brief review of the existing research on conflict detection. We then detail the experimental plan and the assumptions which underpin it. The derivation of the statistical model is then presented and discussed. Need to complete CD&R schemes The difficulty in predicting an insufficient separation between aircraft stems from the fact that a significant amount of air traffic cannot be sorted out in advance into the two cases – conflict and non conflict – as long as the strict separation minima remain the criteria (Alliot, Durand and Granger, 2001). Indeed, a fundamental feature of ATC is that uncertainty vitiates the data required for making conflict diagnoses. This uncertainty includes a large range of different aspects: variability, ambiguity, incomplete or missing data, etc. These come from various (and varying) flight environment factors – wind shift, engine parameters setting by pilots, etc. – which result in inaccurate or even inappropriate conflict prediction. Management of this uncertainty is therefore central to the conflict detection process and appears to differ when it is processed by CD&R systems rather than by using ATCo expertise. Thresholds metrics used in CD&R systems have indeed failed until now to adequately meet results of experts’ heuristics, especially as they cannot easily take into account important cost aspects such as controller’s workload (Kuchar and Yang, 2000). Consequently, the subset of conflicting aircraft coming from ATCo expertise and that coming from system algorithms are not the same in general, and can not be identified with the subset of factual “losses of separation” that would have resulted from the initial conflict contingencies. There is nothing inherently surprising in this difference as, in most cases, algorithms used in "tactical collision alerting systems" ultimately aim to automate the conflict detection and resolution tasks. As such, they are only marginally interested in complying with controllers’ processes. Xu and Rantanen (2003) acknowledge, however, that data on human performance could provide "a guideline for designers to develop and improve automated conflict detection that can off-load the controller's spatial temporal cognition". So, our guess is that a step forward could be made if a conflict detection model that reproduces the judgments of controllers could be proposed. From the workload point of view, this makes perfect sense as conflict-related measures of complexity have persistently been found to predict the controller's workload or assessment of traffic complexity in a number of studies, most of which were reviewed in Hilburn (2004) (see also Kopardekar and Magyarits (2003) for a review of these variables and their integration in a "unified dynamic density metric"). If these variables, built on "objective" (as in Kuchar and Yang, 2000) conflict detections models, do well in predicting workload, why shouldn't analogous variables built upon a "subjective" model of conflict detection perform even better 1 ? A more ambitious application of such a model is provided by the ongoing European Community-funded project ERASMUS: the basic idea of ERASMUS (and so-called subliminal control) is that imperceptible alterations of an aircraft’s speed or climb rate could be used to provide ATCos with "lucky traffic", where conflict contingencies are uncannily but “naturally” rare. Accordingly, the objective of the "subliminal control problem" as presented in Crück and Lygeros (2007) is to "minimize the level of risk that will be perceived by the ATCo, using only imperceptible maneuvers". This requires that the controllers' judgments for any potential conflict situation can be estimated. Resource-consuming conflict situations could therefore be identified and avoided whenever possible. Human-centered analysis of Conflict Detection To build a predictive model of controllers' judgments on conflict detection, it seems necessary to understand how this task is performed in real field operations and to focus on the human centered aspect of conflict detection. Various cognitive processing and strategies are in fact likely to be used by ATCos but their connections with traffic configurations to which they are applied are not clearly established (Xu and Rantanen, 2003) (Loft et al., 2007).The fundamental result on the subject is that experience acquisition by ATCos particularly results in becoming long in heuristics and short in computing within the conflict detection process (Bisseret, 1981). This particular feature of ATC practice is responsible for the fact that a large part of conflict detection is automated. Indeed, perceptual processes underlying detection processes both involve heuristics and visually available data. As airspace structure and flow distribution in each sector determine specific locations and events from which conflicts are more likely to occur (Loft et al., 2007), these processes are basically interpretation ones, automatic, influenced by context and expectation (Willems et al., 1999) (Landry, 1999). A daily repeated practice and the rule-based environment of ATC (published or usual trajectories) therefore allows controllers to substantially automating the detection process (Leroux, 1999). Also, this prevalent use of heuristics is likely to favor the development of inter-individual differences that may well have existed ab initio: Law et al., (1993), for example, show that men and 1 Note that, following this idea, conflict detection measures built around a preliminary version of this work have been included in the set complexity indicators used in Gianazza and Guittet (2006a, 2006b) to model the link between workload and dynamic sectorization in the French airspace. women differ in their ability to process velocities and distances concurrently. This explains why, when asked to estimate conflict risk, controllers put forward partly different judgments (Averty, 2005), which could, a priori, appear unexpected from full performance-level experts. For all that, this variability does not systematically prevent consistent judgments from emerging, since judgments also largely depend on objective features of the configuration of aircraft trajectories (Averty, 2005) (Nunes and Kirlik 2006). Typically, separation assessment on the horizontal plane involves visual heuristics and generally calls for pattern processing (Enard, 1974) (Nunes and Mogford, 2003) (Xu and Rantanen, 2003). Since conflict detection requires controllers to foresee future behavior and positions of aircraft, a projection process (extrapolation) is assumed using both the perceptible current flight data and the individual experiences of operators (Endsley, 1995) (Xu and Rantanen 2003) (Davison 2006) (Boag 2006). Focusing on the former, the separation assessment in the horizontal plane requires the controller to evaluate the “time to conflict”. While the time to the closest point of approach (CPA) is the exact information from which a possible miss distance could be inferred, Xu and al. (2006) sensibly put forward that an intermediate step could exist in such a task, leaning on the 2-D trajectory intersection point (IP), easier to perceptually extrapolate than CPA. Although no evidence exists yet to explain the method they use, it is conceivable that ATCos initially grade conflict contingencies with regard to the mere distance of each target to the IP. This point has been reported to be needed – i.e. extrapolated – by controllers (Enard, 1974) (Bisseret, 1981) (Davison, 2006). The distance of each aircraft to this point is the salient information, biasing judgments (Law et al., 1993). The closest aircraft is indeed preferentially estimated as arriving first over the IP, even for cases where the velocity ratio actually makes the furthest away come first (“distance-over-speed” bias). In fact, the speed ratio has contradictory effects, according to literature (Enard, 1975) (Xu and Rantanen, 2003) and it can be used in ATC for reinforcing a judgment that has already begun to emerge (Bouju, 1978). Furthermore, when present, acceleration is a parameter that controllers perceive with difficulty and do not frequently integrate into projected positions (Davison, 2006). Despite being established from numerical values, and therefore needing some computation, altitude is often found to be the privileged parameter to establish conflict diagnosis (Rantanen and Nunes, 2005) (Loft et al., 2007). This diagnosis is magnified by the fact that numerous studies only involve aircraft flying levels. When vertical separation is not granted, aircraft attitude – i.e. the fact they are level, climbing or descending – seems to have strong effects (Lafon, 1978) (Lamoureux, 1999): except for non-radar separated aircraft, i.e. flying either at different altitudes or on clearly distinct routes, conflict judgment needs information on the four dimensions (spatial and temporal). As soon as one of the involved aircraft is climbing or descending (or supposed to get this attitude in the short or medium term), judgment will depend on whether the aircraft will be found to reach the same altitude at the moment of the position overlap on the horizontal plane (Boag et al., 2006). Rates of climb/descent by themselves are seldom used by controllers, bringing about inaccurate assessment of vertical positions (Michard, 1976), except probably for aircraft of identical attitude (Bouju, 1978). Boag et al. (2006) underlined the link between complexity of conflict detection and the “number of relations” between variables from a pair of aircraft that needed to be processed to perform the diagnosis. They proposed metrics to categorize conflicts according to this relational complexity. Other geometrical features of trajectories also have an impact on judgment accuracy (Remington et al., 2000). An increase in convergence angles generally makes judgments more inaccurate, reaching a maximum effect near 90° (Bouju, 1978) (Nunes and Scholl, 2004). According to Law et al. (1993), an interaction between aircraft speed and convergence angle also seems to exist, impacting the perceived complexity and the accuracy of judgment. Finally, conflict processing and workload have the closest relationship in ATC (Lamoureux, 1999) (Remington et al., 2000) (Averty et al., 2004) (Loft et al., 2007). In particular, conflict classification is radically impacted by contextual workload, since it has been shown that ATCos may sort out a part of the traffic into either conflict or non conflict with the aim of managing their own workload – a phenomenon of workload homeostasis by strategy change (Sperandio, 1978). Actually, maximizing the relevance of diagnoses is time/resources consuming (Leroux, 1999) and leads to early defusing of the current conflict possibilities in order to regulate workload. This basically means increasing the number of conflict diagnoses. Thus, “diagnosing conflicts” does not uniquely depend on the configuration of the involved aircraft and their intrinsic separation probability. Integrating doubt management in Conflict Detection A large part of research on human-centered conflict detection focuses on the performance of the subjects, either through the accuracy of the diagnosis or by the time needed to formulate them. Some authors, however, acknowledge that conflict judgment inherently includes a doubt dimension (Enard, 1974) (Bisseret, 1981) (Leroux, 1999). Bisseret (1981) even showed that the precision of the calculations is not of prime importance for the controller. It is indeed far less detrimental for him to accept a certain degree of imprecision if it gives him better guarantees against omissions of conflict diagnosis, than to use calculations in order to reduce uncertainty to a maximum, and be "surprised" from time to time by a diagnostic error. As experience is acquired, computation processes step aside in favor of more holistic but robust processes, capable of preventing the cognitively disruptive effects of certain events. Consequently, maintaining the doubt in his/her mental representation is very profitable to the ATCo. By deferring the decision to classify a certain traffic configuration into conflict or no conflict for as long as is reasonable, the controller minimizes the costs of an error of appreciation. Furthermore, she can more easily approach the resolution that is necessary and sufficient. This phase of doubt is fundamental in ATC – air traffic control is the “art of doubting” (Leroux, 1999). In other words, it is the management of this doubt, i.e. the decision to prolong it or, on the contrary, to suppress it, which makes up the heart of know-how in ATC. In line with Bisseret (1981) and Leroux (1999), the acquisition of expertise by controllers leads them to substitute computing and accuracy in conflict detection in favor of doubt, resulting in a more robust diagnosis, which is globally equivalent in accuracy since false responses decrease. Our purpose was then to draw an accurate picture of the detection capacity of ATCos, in order to model their real conflict judgments. Consequently, we made the following experiment comply with four main features: To keep only the number of task components that could be experimentally handled. In practical terms, only realistic uncertain data were kept (from the §b above) and a single fixed geometrical configuration was considered. To consider a simple (pair-wise) configuration of aircraft. To present traffic scenarios dynamically, using devices (radar screen and maps, flight strips) peculiar to each center and flights coming from real local recordings. To integrate doubt expression in responses – doubt being viewed as self-assessed reliability on each individual judgment e.g. as a part of ATC expertise. To sum up, our objective may be restated in the taxonomy proposed by Kuchar and Yang (2000): we want to build a three dimensional Conflict Detection model where state propagation is nominal (aircraft are assumed to follow their "maximum likelihood trajectory") and management of uncertainty is left to the controller. The literature on controller centered issues therefore provides a guideline for modeling in that it indicates which factors are most likely to impact judgments and how they should be modeled.